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ABSTRACT 


The  research  reported  here  examines  overall  training  effective¬ 
ness  for  U.S.  Naval  personnel;  the  measure  of  effectiveness  comes 
from  the  survey  responses  of  the  supervisors  of  first-term  enlisted 
Naval  personnel ,  These  supervisors  were  asked  about  the  productivity 
of  "typical"  first-term  personnel  who  were  schooled  in  the  classroom 
or  trained  on  the  job.  The  survey  is  the  Enlisted  Utilization  Survey 
(EUS)  conducted  by  the  RAND  Corporation  for  the  Department' of  Defense 
in  the  mid-seventies.,  The  EUS  included  over  2,000  supervisors' 
productivity  assessments  for  twelve  Navy  occupational  categories: 
seven  of  the  specialties  offer  the  alternative  of  formal  schooling  or 
on-the-job  training.  For  five  of  the  specialties;  tmly  formal  class¬ 
room  training  is  used.  Overall^  these  specialties  characterize  a 
broad  cross-section  of  Navy  jobs  that  vary  considerably  in  technical 
complexity  and  formal  school  length. — 

The  aata  permit  us  to  estimate  learning  curves  (the  growth  of 
productivity  or  effectiveness)  for  both  training  methods.  The 
measure  of  effectiveness  is  the  net  productivity  of  the  “typical" 
trainee,  that  i3,  the  contribution  of  the  trainee  minus  the  loss  in 
production  of  experienced  personnel  who  must  supervise  and  train 
him.  (This  is  the  implicit  cost  of  on-the-job  training.)  For  the 
cost  of  formal  schooling,  we  used  Navy  administrative  data. 
Information  about  the’  performance  of  the  typical  first-termer,  for 
both  modes  of  training  was  available  at  four  time  periods:  after 
1  month,  I  year,  2  years,  and  4  years  at  his  first  duty  station. 
Comparisons  are  drawn  between  the  two  training  methods.  Finally/ the 
usefulness  of  research  on  training  effectiveness  in  other  areas  of* 
military  manpower  research  is  discussed. 
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AN  EVALUATION  OF  THE  EFFECTIVENESS  OF  CLASSROOM 
AND  ON-THE-JOB  TRAINING 


1.  INTRODUCTION 

,  The  value  of  different  methods  of  training  military  personnel 
depends  on  both  their  costs  and  effectiveness.  Without  such  infor¬ 
mation,  training  methods  and  choices  among  them  cannot  be  evaluated 
objectively.  Given  the  importance  of  information  on  training  costs 
and  the  subsequent  effectiveness  of  different  training  modes,  the 
paucity  of  research  is  surprising.  The  reasons,  however,  are  not 
difficult  to  uncover.  First  there  is  the  difficulty  of  measuring  the 
effectiveness  of  the  training.  Pen-and-paper  or  hands-on  tests  have 
been  used  to  determine  how  well  particular  skills  were  learned,  but 
there  is  little  agreement  on  a  metric  for  translating  test  scores 
into  on-the-job  effectiveness.  Overall  comparisons  of  competing 
training  methods  (conventional  classroom  instruction  versus  training 
done  exclusively  on  the  job)  have  been  hampered  by  the  difficulty  of 
measuring  effectiveness  and  by  the  difficulty  of  measuring  training 
costs  when  training  is  done  exclusively  on  the  job.  A  substantial 
gap  exists  between  the  resources  devoted  to  the  training  of  military 
personnel  and  the  resources  devoted  to  evaluating  that  training.* 

To  help  close  this  gap,  the  research  reported  here  examines  the 
effectiveness  of  training  for  U.S.  Naval  personnel,  using  data  from 
the  Enlisted  Utilization  Survey  and  from  Navy  administrative 
records.  Using  these  sources,  we  assess  the  productivity  of  typical 
first-term  Naval  enlistees  in  12  major  occupational  specialties,  by 
occupation  and  type  of  training,  at  four  points  in  time.  We  also 
construct  cost  estimates  for  the  two  training  modes. 

After  estimating  effectiveness  growth  curves,  we  construct 
indices  for  the  fraction  of  an  average  careerist's  output  (for  each 
specialty  and  training  mode)  produced  by  first-term  Individuals. 
Finally  we  integrate  Information  on  formal  schooling  costs  with  the 
effectiveness  data,  and  compare  the  cost  effectiveness  of  the 
alternative  training  methods. 

2.  MEASURING  THE  EFFECTIVENESS  OF  TRAINING 

An  issue  frequently  confronted  by  military  manpower  analysts  is 
the  relative  effectiveness  of  first-term  and  career  personnel. 
Similarly,  there  are  questions  about  the  efficacy  of  the  different 
methods  of  training  recruits.  To  address  such  concerns,  one  needs  to 


*  For  an  example  of  some  important  earlier  work  and  a  discussion  of 
the  difficulties  in  obtaining  robust  estimates,  see  [1]. 


quantify  the  relative  effectiveness  of  individuals  in  different 
categories,  whether  the  categories  be  recruits  trained  by  different 
techniques  or  first-termers  versus  more  experienced  military 
personnel. 

Quantification  of  productivity  or  ef fectivenesss  is  complicated 
by  the  fact  that  individuals  within  any  particular  occupation  perform 
many  tasks:  an  individual  may  excel  at  some  of  these  tasks  but  per¬ 
form  others  less  well.  Summary  measures  of  overall  effectiveness  are 
required,  however,  if  unambiguous  comparisons  are  to  be  drawn  across 
different  training  modes.  In  this  paper  we  have  chosen  personnel 
assessments  made  by  supervisors  as  our  measure  of  overall  effective¬ 
ness.  In  particular,  we  use  supervisors’  responses  from  a  survey 
conducted  by  the  RAND  Corporation  for  the  Defense  Advanced  Research 
Projects  Agency.  Let  us  now  turn  to  a  discussion  of  this  survey. 

2.1  The  Enlisted  Utilization  Survey  (EUS) 

The  EUS  surveyed  first-term  personnel  at  their  first  duty 
station  between  November  1974  and  January  1975  in  58  Army,  Navy,  and 
Air  Force  occupational  specialties.*  The  survey  was  done  in  two 
parts.  The  first  questionnaire  went  to  enlistees,  asking  for  some 
background  information  and  the  names  of  up  to  three  immediate 
supervisors.  The  second  questionnaire  went  to  the  identified  super¬ 
visors,  seeking  productivity  information  on  the  recruit  and  some 
background  information  on  the  supervisor.  The  responses  to  the  two 
questionnaires  were  then  merged  with  individual  personnel  files  from 
the  appropriate  service. 

Here  we  analyze  only  Chose  EUS  data  that  pertain  to  the 
Navy.**,  For  Naval  personnel,  there  are  over  2,000  supervisors' 
productivity  assessments  for  15  Navy  occupational  categories.  The 
15  occupational  categories  cover  about  50  perce  -t  of  Navy  enlisted 
personnel.  Three  of  the  categories,  however,  comprise  general-duty 
personnel  for  which  no  training  (other  than  initial  recruit  training) 
is  provided;  enlistees  in  these  specialties  will  be  discussed  in  a 


*  See  [2],  [3],  and  14]  for  a  detailed  discussion  of  these  data  and 
of  the  usefulness  of  survey  data  in  general. 

**  Although  the  RAND  researchers  focused  on  the  Air  Force  personnel 
data,  they  provided  descriptive  statistics  for  the  overall  sample; 
the  Navy  data,  however,  were  never  extensively  analyzed. 
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separate  report.*  Seven  of  the  occupations  offer  a  choice  of 
training  methods — formal  schooling  (A-school)  or  on-the-job  training 
(OJT)  •  Training  for  five  of  the  jobs  is  done  only  in  school  (three 
of  these  jobs  are  in  the  nuclear  power  area).  Overall,  the  occupa¬ 
tions  we  analyze  cover  a  broad  cross-section  of  Navy  jobs  that  vary 
considerably  in  technical  complexity  and  length  of  formal  schooling. 

The  EUS  asked  supervisors  to  rate  the  net  contribution  to  output 
of  first-term  individuals  at  several  points  within  the  first  enlist¬ 
ment  term.  The  productivity  assessments  are  all  relative  to  an 
average  specialist  with  4  years  of  experience  in  that  occupation.  In 
this  paper  we  focus  on  supervisors'  assessments  of  the  effectiveness 
of  "typical"  individuals  in  each  training  path;  the  assessments  are 
at, four  different  points  during  the  first  enlistment  term.  Our 
particular  focus  is  Navy  occupations  for  which  training  can  take 
place  either  exclusively  on  the  job  or  through  a  combination  of. 
formal  schooling  and  on-the-job  training. 

2.2  The  Growth  of  Productivity  in  the  First  Tern 

The  EUS  questionnaire  distributed  to  sr^rvisors  asked  the 
respondents  to  rate  the  relative  productivity  of  typical  first-term 
enlistees  in  their  occupation  1  month  after  arriving  at  the  first 
duty  station.  Separate  questions  were  asked  about  the  relative 
productivity  of  a  typical  graduate  from  A-schcol  and  a  typical 
direct-duty  assignee.** 

The  survey  attempted  to  assess  net  productivity  of  the  trainee, 
that  is,  the  contribution  of  the  trainee  minus  the  loss  in  production 
of  experienced  personnel  who  must  train  and  supervise  him.  At  the 
1-month  point,  the  trainee's  net  contribution  to  unit  output  is  often 


*  Productivity  results  for  Airmen,  Seamen,  and  Firemen  (who-  do  not 
achieve  or  "strike  for"  a  specific  occupation)  are  not  anaxyzed  In 
this  paper,  which  focuses  on  the  growth  of  productivity  for  individ¬ 
uals  who  become  occupationally  qualified  petty. officers  during  the 
first  4  years.  It  should  be  understood,  however,  that  a  discussion 
of  the  product,  ity  of  a  typical  on-the-job-trained  petty  officer  . 
involves  a  period  of  time  when  he  was  not  occupationally  designated 
(a  non-designated  striker).  The  promotion  steps  for  an  Aviation 
Machinist's  Mate  (AD),  as  an  example,  entail  being  advanced  to  Airman 
(AN),  then  becoming  a  designated  striker  (ADAN),  and  finally  being 
promoted  to  the  occupationally  qualified  level,  petty  officer  third 
class  (AD3). 

**  Other  work  by  the  authors  analyzed  survey  results  for  individual 
recruits.  This  paper,  however,  focuses  on  the  supervisors'  responses 
to  questions  concerning  the  "typical"  recruit. 
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negative.*  Performance  of  a  trainer  was  also  evaluated  after  1  year, 
2  years,  and  4  years  at  a  duty  station. 

In  particular,  for  A-school  graduates,  the  questionnaire  asked 
the  supervisor  to  “estimate  the  typical  new  'A'  school  graduate's  NET 
CONTRIBUTION  TO  UNIT  PRODUCTION  at  several  points  in  his  service 
career,  assuming  he  serves  4  years  or  more  in  this  shop  or  section. 

An  individual's  NET  CONTRIBUTION  TO  UNIT  PRODUCTION  is  his  direct 
production  minus  production  lost  by  others  who  supervise  and  instruct 
him."  The  assessment  was  further  directed  to  be  "relative  to  the 
average  specialist  with  four  years  experience." 

The  averages  of  the  supervisors'  responses  (for  each  of  the  four 
points  in  time)  are  presented  in  table  1.  First,  the  results  are 
shown  for  occupations  that  have  alternative  training  paths.  Next, 
results  are  reported  for  occupations  trained  exclusively  in  A-school 
(three  nuclear  power  fields  and  two  medical  specialties). 

The  general  results  of  table  1  are  in  accord  with  our  expecta¬ 
tions.  First,  in  all  occupations  average  productivity  grows  over 
time.  Second,  in  occupations  that  offer  alternative  training  paths, 
the  productivity  of  A-school  graduates  exceeds  that  of  those  learning 
exclusively  on  the  job.  Since  A-school  graduates  have  spent  142  to 
468  days  in  the  classroom  learning  the  required  skills  before  they 
arrive  at  their  duty  station,  a  different  result  would  be  surprising. 
Third,  the  typical  OJT  trainee  never  reaches  the  4-year  average 
specialist  level.  Although  this  result  is  not  inevitable,  it  is 
plausible:  the  dominant  training  mode  used  by  the  Navy  fcr  these 
specialists  is  A-school.  Finally,  average  productivity  after  4  years 
at  the  duty  station  is  approximately  100  percent  for  A-school 
attendees.  Since  the  productivity  estimate  is  inherently  normed— the 
-ypical  enlistee  in  the  specified  training  path  relative  to  the 
average  fourth-year  specialist — this  result  again  conforms  to 
expectations. 

Results  for  three  Navy  occupations— Electricians'  Mates  (EMs), 
Machinists'  Mates  (MMs)  and  Aviation  Electricians'  Mates  (AEs) — are 
presented  in  figures  1,  2,  and  3.  The  illustrations  make  a  clear 
point  of  how  much  of  the  first  enlistment  period  is  spent  learning. 

The  productivity  estimates  presented  in  table  1  are  point-in- 
tlme  estimates.  To  convert  these  estimates  into  a  continuous  measure 


*  The  advantages  of  having  a  measure  of  net,  rather  than  gross, 
productivity  are  enormous.  Without  a  measure  that  includes  the  loss 
of  productive  time  by  supervisory  personnel,  it  would  not  be  possible 
to  calculate  the  cost  of  on-the-job  training.  In  short,  cost-benefit 
comparisons  of  the  two  training  modes  would  not  be  possible.’ 
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Percent-of  output  of  •  4-year  specialist  22  Percent  of  output  of  4-year  specialist 


!.  1:  PRODUCTIVITY  GROWTH  FOR  ELECTRICIANS 
MATES  IN  THE  FIRST  TERM 


FIG.  2 
ELEl 


PRODUCTIVITY  GROWTH  FOR  AVIATION 
CTRICIANS'  MATES  IN  THE  FIRST  TERM 


FIG.  3:  PRODUCTIVITY  GROWTH  FOR  MACHINISTS' 
MATES  IN  THE  FIRST  TERM 


of  productivity  growth  over  time,  the  following  equation  was 
estimated  for  each  occupational  specialty: 

NP  *  a  +  bjt  +  b2t^  .  . 

NP  ,  is  the  value  of  net  productivity  and  t  is  the  time,  in  months,  at 
the  duty  station.*  The  regression  results  by  rating  and  by  training 
type  are  reported  in  appendix  A. 

The  average  productivity  values  and  the  estimated  regression 
coefficients  provide  information  about  the  value  of  a  first-termer, 
but  there  are  too  many  numbers  to  facilitate  easy  comparisons  across 
training  modes.  The  next  step,  therefore,  is  to  develop  a  single 
measure  of  effectiveness  or  productivity  for  each  training  mode.  As 
our  aggregate  measure,  we  use  the  value  of  a  first-term  recruit 


*  In  the  regressions,  time  is  measured  as  months  at  the  duty  station 
(not  time  in  the  Navy).  This  procedure  was  computationally  simpler. 
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r.  relative  to  the  value  of  an  average  specialist  with  4  years  of 

J  experience.  The  following  section  describes  the  two  indices  we 

constructed. 

'  3.  INDICES  OF  FIRST-TERMER  PRODUCTIVITY 

IHere  we  want  to  construct  indices  that  measure  the  productivity 
of  typical  first-termers  relative  to  the  productivity  of  trained 
specialists.  Our  definition  of  a  trained  specialist  comes  from  the 
EUS  questionnaire,  namely  4  years  of  work  experience  in  the  special¬ 
ty.  The  first  index  is  the  average  fraction  of  the  output  of  a 
traihed  specialist  that  is  produced  during  the  first  4  years  in  the 
f*  Navy.  Graphically,  it  is  the  shaded  area  in  figure  4  (minus  the 

14  hatched  area  where  net  productivity  is  negative). 


2 


£ 


More  formally. 


FIG.  4:  HYPOTHETICAL  MEASUREMENT 
OF  FIRST-TERMER  PRODUCTIVITY 


I 


P(t)dt 


/48  , 


where 


tQ  is  the  month  of  arrival  at  first  duty  station* 

P(t)  is  productivity  at  time  t.  (Productivity  is  measured  as 
the  fraction  of  the  output  of  a  fourth  year  specialist  that  is 
produced.) 

This  index  allows  comparisons  of  relative  productivity  during  the 
first  4  years  for  different  training  modes  within  an  occupation. 
Additionally,  it  allows  comparisons  across  different  Navy  occupa¬ 
tions;  here  it  facilitates  understanding  of  how  much  of  the  first 


II 


term  is  spent  in  "learning"  and  how  much  is  spent  in  "doing"  for  the 
different  Navy  jobs. 


The  second  index  differs  in  that  it  looks  at  the  relative 
productivity  for  4  years  after  arrival  at  the  first  duty  station. 
Since  first-termers  in  different  occupations  (or  in  different 
training  paths  within  an  occupation)  spend  different  amounts  of  time 
in  training  before  arriving  at  their  first  duty  station,  this  index 
is  not  quite  so  convenient  for  comparison  across  occupations. 
Specifically,  the  formula  for  the  second  index  is: 


+  48 


P(t)dt 


/(48  +  t  ) 

u 


The  advantage  of  this  index,  however,  is  that  it  utilizes  all  the 
information  in  the  questionnaire:  namely,  the  growth  of  productivity 
for  4  years  after  arrival  at  the  first  duty  station.  Since  the 
second  index  measures  productivity  over  a  longer  time,  capturing  more 
productive  time,  it  is  always  larger  than  the  first  index.*.  Except 
for  the  nuclear  power  occupations  (in  which  training  time  is  very 
long) ,  both  indices  order  the  jobs  in  a  similar  fashion. 


The  resulting  relative  productivity  assessments  for  A-school 
graduates  are  presented  in  table  2.  For  Electronics  Technicians ,  for 
example,  typical  A-school  graduates  average  (over  the  4-year  period) 
41.5  percent  of  the  output  of  a  4-year  specialist's  output  during 
4  years.  If  we  look  at  their  output  over  the  longer  period  encom¬ 
passed  in  the  second  index,  they  produce  51.7  percent  of  the  output 
of  a  specialist  with  4  years  of  experience.  Here  we  are  looking  at  a 
period  of  over  5  years;  supervisors  evaluate  the  average  output  of 
two  first-termers  during  this  period  as  roughly  equivalent  to  the 
output  of  one  experienced  Individual. 

For  the  seven  relevant  occupations ,  table  3  contrasts  the 
A-school  and  QJT  training  paths.  Not  surprisingly,  A-school  gradu¬ 
ates  generate  sore  ner output  over  the  first  4  years  than  those 
trained  solely  on  the  job.  These  comparisons,  however,  are  somewhat 
misleading.  They  account  for  the  full  cost  of  training  only  for 
those  trained  exclusively  on  the  job.  (For  these  trainees  the  cost 
is  the  output  .foregone  because  of  supervisory  time;  our  productivity 
measure  accounts  for  these  costs.)  For  those  trained  in  A-school, 
however,  we  do  not  yet  Include  any  estimates  of  the  formal  schooling 
cost.  In  the  next  section,  we  will  provide  comparisons  that  take 
into  account  the  costs  of  formal  schooling. 


*  Four  years  at  the  duty  station  implies  from  50-1/2  to  64  months  in 
the  Navy  (the  latter  is  for  nuclear-trained  Electronics  Technicians; 
see  appendix  B) . 
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TABLE  2 


PRODUCTIVITY  OF  FIRST-TERM  A-SCHOOL  GRADUATES 
RELATIVE  TO  4- YEAR  SPECIALISTS 

Percent  of  output  of  4-year 
specialist  produced  by 
_ first-termers _ 

Non-nuclear 


occupations 

Index  1 

Index 

ET 

41.5 

51.7 

AE 

45.7  . 

52.7 

MM 

47.8 

52.3 

EM 

49.8 

55.6 

RM 

51.9 

56.5 

MS 

55.9 

58.4 

AD 

55.5 

60.2 

HM 

59.6 

63.6 

DT 

66.5 

69.9 

Nuclear  power 
occupations 

ET 

29.6 

48.2 

MM 

32.0 

43.4 

EM 

37.8 

51.1 
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TABLE  3 


PRODUCTIVITY  OVER  4  YEARS: 
COMPARISON  OF  A-SCHOOL  AND  OJT  TRAINING  PATHS 


Occupations  Index  1 


ET  (A-school) 

41.5 

ET  (OJT) 

34.0 

AE  (A-school) 

45.7 

AE  (OJT) 

32.4 

MM  (A-school) 

47.8 

MM  (OJT) 

41.2 

EM  (A-school) 

49.8 

EM  (OJT) 

35.4 

RM  (A-school) 

51.9 

RM  (OJT) 

41.9 

AD  (A-school) 

55.4 

AD  (OJT) 

42.3 

MS  (A-school) 

55.9 

MS  (OJT) 

47.7 

Productivity  of  A-school 
graduate  relative  to 
direct-duty  assignee 

1.22 

1.41 

1.16 

1.41 

1.24 

1.31 

1.17 


i4 


4.  COST-EFFECTI /ENESS  COMPARISONS 

The  net  productivity  indices  wa  constructed  measured  the  frac¬ 
tion  of  an  average  4-year  specialist's  output  produced  by  individuals 
in  the  first  term.  Now,  we  want  to  integrate  the  information  on  the 
output  of  typical  first-termers  with  information  on  their  costs. 
Appendix  B  provides  detailed  information  on  the  cost  computation. 

All  costs  are  for  FY  1979  and  in  FY  1979  dollars.*  The  data  on 
A-school  costs  came  from  the  Navy's  Training,  Analysis  and  Evaluation 
Group.  (TAEG)  [5]  and  are  reported  in  [6].  A-school  costs  Include 
both  formal  school  costs  and  wage  costs.  They  represent  the  cost  of 
an  A-school  graduate  in  the  appropriate  course.**  Wage  costs  after 
training  are  based  on  FY  1979  pay  rates.  The  cost  of  boot  camp, 
including  wages  to  recruits,  was  estimated  at  $2,165  per  recruit. 

Table  4  details  these  costs.  Personnel  who  train  exclusively  on 
the  job  cost  $23,160  (in  1979  dollars)  over  a  4-year  enlistment 
term.***  Recruits  who  attend  a  formal  school  program  cost  from 
$26,990  for  Mess  Management  Specialist  to  $49,678  for  Nuclear-Power 
Electronics  Technicians. 

The  second  column  of  table  4  reproduces  the  first  productivity 
index  from  table  3  (an  index. of  66.5  for  DTs  means  that  in  the  first 
enlistment  period  a  typical  DT  averages  66.48  percent  of  the  output 
that  would  be  produced  by  a  fully  trained  specialist).  The  final 
column  of  table  4  computes  cost-effectiveness  ratios  for  average 
personnel  in  each  category.  These  are  most  meaningful  for  the  seven 
ratings  in  which  there  are  two  training  paths.  AEs,  for  example, 
cost  $637  per  unit  of  benefit  for  a  typical  A-school  graduate  and 
$714  per  unit  of  benefit  for  a  typical  non-A-school  graduate. 

Although  the  results  are  somewhat  mixed,  our  estimates  indicate 
that  for  most  occupations,  graduates  of  formal  schooling  are  more 
cost  effective  than  those  trained  exclusively  on  the  job.  This  is 
true  even  though  chose  trained  in  schools  have  considerably  higher 
explicit  costs  than  those  trained  exclusively  on  the  job.  In  short, 
the  greater  productivity  of  A-school  graduates,  even  over  this  rela¬ 
tively  short  period,  appears  to  make  this  training  mode  a  better  buy. 


*  Transforming  Navy  cost  figures  for  individual  courses  into  cost 
figures  for  occupational  qualification  is  a  non-trivial  task;  here  we 
use  data  constructed  for  another  study,  [6]. 

**  If,  for  example,  10C  individuals  begin  the  course  but  only  80 
finish  it,  the  schooling  costs  are  spread  over  the  80  graduates. 

***  Neither  cost  nor  productivity  Information  is  discounted;  we  are 
analyzing  Steady-state  alternatives. 
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TABLE  4 


COST  EFFECTIVENESS  COMPARISONS  FOR  4  YEARS  OF  SERVICE 


. 

I 

Total  cost 
(from 
table  B-l ) 

II 

Effectiveness 
(Index  1) 

HI 

Dollar  cost  per  unit  of 
effectiveness  (column  I 
divided  by  column  II) 

Non-nuclear 

occupations 

MS  (A-school) 

$26,990 

55.9 

482 

MS 

(OJT) 

23,160 

47.7 

486 

AD 

(A-school) 

26,637 

55.4 

481 

AD 

(OJT) 

23,160 

42.3 

547 

MM 

(A-achool) 

28,843 

47.8 

604 

MM 

(OJT) 

23,160 

41.2 

562 

HM 

(A-school) 

28,431 

59.6 

477 

DT 

(A-school) 

28,999 

66.5 

436 

RM 

(A-school) 

29,456 

51.9 

567 

RM 

(OJT) 

23,160 

41.9 

553 

EM 

(A-school) 

29,137 

49.8 

585 

EM 

(OJT) 

23,160 

35.4 

654 

AE 

(A-school) 

29,090 

45.7 

637 

AE 

(OJT) 

23,160 

32.4 

714 

ET 

(A-school) 

35,598 

41.5 

859 

ET 

(OJT) 

23,160 

34. C 

682 

Nuclear  power 
occupations3' 

MM 

43,055 

34.0 

1,264 

EM 

43,332 

37.8 

1,147 

ET 

49,678 

30.6 

1,623 

aThe3e  occupations  have  an  initial  obligation  of  6  years. 
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In  two  occupations,  Radiomen  and  Machinists'  Mates,  on-the-job 
trainees  are  slightly  more  cost  effective.  Only  for  Electronics 
Technicians  does  the  CUT  training  path  really  appear  superior.  This 
result,  however,  may  reflect  the  fact  that  most  ETs  are  6-year 
obligors,  and  the  4-year  period  over  which  we  evaluate  benefits  and 
costs  is  probably  too  short.  Even  though  ETs  trained  in  A— school  are 
more  productive  than  those  trained  on  the  job,  the  fact  that  the 
first  year  and  a  half  is  spent  in  formal  schooling  makes  their  pro¬ 
ductivity  quite  low  until  the  fourth  year.  Therefore,  for  6-year 
obligors  (Electronics  Technicians  and  the  three  nuclear  power  occupa¬ 
tions)  the  calculations  should  be  done  over  a  6-year  period. 
Unfortunately,  since  the  survey  normed  productivity  data  for  the 
4-year  specialist,  we  do  not  have  the  information  to  make  such 
calculations. 

In  a  larger  sense,  focusing  only  on  the  productivity  of  first- 
termers  may  unfairly  irflate  the  cost-to-benef it  ratio  for  those 
trained  in  formal  school  programs.  First,  in  addition  to  their 
explicit  costs;  these  programs  consume  a  considerable,  amount  of  time 
within  the  first  term.  Even  though  the  formal  schooling  alternative 
for  RMs,  MMs,  and  ETs  did  not;  appear  cost  effective  over  the  first 
4  years,  an  8-year  time  horizon  might  provide  a  different  picture. 

To  test  this  conjecture,  additional  data  would  be  required.  Indeed, 
we  would  suggest  that  there  is  a  clear  need  for  more  extensive 
collection  of  these  types  of  data,  a  point  to  which  we  return  later 
in  this  paper.  First,  however,  let  ,us  discuss  the  uses  for  findings 
like  these  on  training  effectiveness. 

5,  THE  USEFULNESS  OF  THESE  FINDINGS  IN  OTHER  AREAS  OF  MANPOWER 
ANALYSIS 

Findings  on  training  effectiveness  can  be  used  to  study  many 
aspects  of  military  manpower — occupational  assignments,  the  timing  of 
training,  the  length  of  initial,  enlistment  contracts,  and  the  rela¬ 
tive  value  of  accession  and  retention.  Some  of  these  applications 
have  already  been  undertaken,  some  are  forthcoming,  and  others  await 
more  detailed  data.  Because  we  believe  it  is  critical  to  incorporate 
Information  on  both  training  effectiveness  and  the  time-path  of 
learning  into  other  areas  of  manpower  analysis  (some  of  which  may  at 
first  appear  quite  distinct  from  training  analysis),  we  shall  discuss 
each  of  these  topics  in  turn. 

5.1  Occupational  Assignments  in  the  Military 

One  important  problem  faced  by  the  U.S.  military  is  the  high  . 
rate  of  attrition  within  the  first  enlistment  term.  A  study  by 
Thomason  [7]  has  suggested  that  first-term  attrition  could  be  reduced 
if  better  initial  occupational  assignment  policies  were  followed. 

This  study  collected  occupatioa-specif ic  attrition  rates  for 
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individuals  in  different  age  groups  and  education  categories.  Tien, 
individuals  were  reassigned  to  occupations  so  as  to  maximize  first- 
term  retention;  the  overall  gain  in  retention  was  about  10  percentage 
points. 

Although  Thomason's  study  made  an  important  contribution  to  our 
understanding  of  personnel  management,  we  believe  that  it  is  but  a 
first  step  in  analyzing  efficient  occupational  assignment.  The  data 
that  were  available  precluded  using  occupational  assignment  to 
maximize  the  cost-benefit  ratio  for  the  first-term  cohort.  Instead, 
the  fraction  of  the  original  cohort  that  remained  until  the  end  of 
the  first  term  was  maximized  (i.e.,  retention). 

In  Thomason’s  formulation,  It  is  immaterial  whether  an  individ¬ 
ual  drop  out  6f  the  Navy  just  before  formal  schooling,  or  just  after 
formal  schooling,  or  with  1  day  left  on  his  first-term  commitment: 
all  of  these  individuals  were  identified  as  dropouts.  Our  data 
suggest  that  it  makes  a  substantial  difference  when  an  individual 
leaves  within  the  first  term.  Data  on  the  costs  of  formal  schooling 
given  in  appendix  C  suggest  that  several  thousand  dollars  of  savings 
are  associated  with  attrition  before,  rather  than  after,  attending 
the  Navy' 8  A-school  program.  Clearly  there  is  a  need  to  integrate 
cost-benefit  data  with  attrition  data  if  the  Navy  is  to  efficiently 
manage  its  occupational  pipelines.  A  more  ambitious. effort  in  this 
area  has  the  potential  to  produce  considerable  cost  savings. 

5.2  The  Timing  of  Training 

Another  training  issue  is  the  question  of  the  timing  of  training 
within  an  occupational  category.  In  the  U.S.  Navy  this  usually 
involves  questions  about  whether  an  individual  should  go  directly 
from  boot  camp  (recruit  training)  to  occupational  training  school  or 
whether  he  should  first  have  some  experience  with  the  fleet.  Funda¬ 
mental'  to  these  discussions  is  the  principle  that  if  the  individual 
is  not  suited  to  Navy  life,  it  is  better  that  he  leave  the  Navy 
before  the  expenditure  of  training  dollars. 

Unfortunately,  this  is  an  area  in  which  we  have  very  litle  data 
to  analyze.  We  simply  do  not  know  how  the  timing  of  training  affects 
attrition,  or  whether  the  current  training  pipeline  patterns  are 
efficient  or  not.  Part  of  the  gap  results  from  the  lack  of  data 
bases  that  Integrate  information  on  the  timing  of  training  with 
individual  personnel  files.  It  is  extremely  time-consuming  (and 
expensive)  to  build  such  data  bases;  in  our  judgement,  however,  it  is 
worthwhile  to  do  so. 
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5.3  The  Length  of  the  Initial  Enlistment  Contract 


Historically,  the  length  of  military  contracts  has  varied 
widely.  During  the  Revolutionary  War,  contracts  for  American  militia 
men  averaged  less  than  3  months;  in  contrast,  the  contracts  for 
recruits  in  the  British  Navy  are  currently  for  20  years,  although  an 
individual  can  give  18-months  notice  after  a  specified  period  has 
passe  . 

Before  the  advent  of  the  All  Volunteer  Force  in  1973,  initial 
active-duty  enlistment  contracts  averaged  2.8  years  in  the  U,S.; 
currently  the  average  is  3.7  years.  While  the  majority  of  the  con¬ 
tracts  are  for  4  years,  in  FY  1980,  7  percent  of  the  contracts  were 
for  more  than  4  years,  and  39  percent  were  for  less  than  4  years. 

The  Army  has  the  shortest  average  contract,  with  over  two-thirds  of 
its  initial  contracts  for  3-year  periods.  Only  the  Air  Force  and  the 
Navy  use  initial  contracts  longer  than  4  yerrs.  The  Navy,  of  all  the 
services,  has  the  greatest  variability  in  the  length  of  an  Initial 
active-duty  obligation,  requiring  enlistments  of  5  or  6  years  for 
occupational  specialties  with  larger  training  components,  and 
experimenting  with  2-  and  3-year  contracts  for  general- detail 
sailors. 

Against  the  backdrop  of  this  variability  in  length  of  the  ini¬ 
tial  enlistment  period,  there  have  been  suggestions  that  the  initial 
enlistment  period  be  lengthened  in  all  services  [8],  Longer  ^ 
enlistment  terms,  of  course,  reduce  the  required  number  of  yearly 
accessions  and  the  number  of  Individuals  who  must  be  trained.  Longer 
contracts  generate  cost  savings  by  reducing  instructional  cost  and 
releasing  personnel  and  equipment  from  instructional  tasks. 

t  1 

Clearly,  however,  it  is  not  costless  to  extend  the  enlistment 
period.  To  make  a  longer  contract  attractive,  even  if  required 
accessions  are  reduced,  probably  involves  an  increase  in  military 
pay.  Indeed,  the  proposal  only  makes  sense  if  the  required  pay 
Increases  are  smaller  than  the  savings  achieved  by  having  to  train 
fewer  individuals.  For  military  occupations  with  short  training 
times,  cost  savings  will  be  small.  Similarly,  occupations  with  long 
training  times  will  achieve  larger  cost  savings  If  fewer  recruits 
need  to  be  trained.  The  observation  that  cost  savings  will  vary 
across  occupations  suggests  it  is  unlikely  that  all  military 
occupations  should  have  the  same  contract  length.  In  fact,  the 
current  variety  in  the  length  of  the  initial  active-duty  enlistment 
appears  to  reflect  differences  in  Training  costs:  occupations  with 
smaller  training  components  have  shorter  contracts  than  those  with 
larger  training  components.  . 

How,  then,  have  current  contract  lengths  been  sec?  Have 
enlistment  lengths  been  systematically  established  so  that  military 


output  is  obtained  at  least  cost?  Given  the  importance  of  the  issue 
and  the  variety  in  initial  contract  periods,  there  is  surprisingly 
little  rigorous  analysis. 

Available  data  can  be  used  to  evaluate  current  first-terra 
contract  lengths  and  to  estimate  the  effects  on  cost  and  output  of 
changes  in  the  contract  period.  Some  assumptions  will  have  to  be 
made  because  existing  data  do  not  allow  empirical  estimation  of  all 
the  important  variables.  However,  existing  information  should  permit 
a  worthwhile  examination  of  the  costs  and  benefits  of  varying  first- 
term  enlistment  lengths.  Such  an  effort  is  underway  [9,  101. 

5.4  Accession  Versus  Retention 

One  problem  that  the  U.S.  Navy  has  faced  for  many  years  is  a 
shortage  of  mid-career  petty  officers.  There  are  two  principal  ways 
to  alleviate  this  shortage:  recruit  more  personnel  initially  or  . 
encourage  more  personnel  to  stay.  The  first  strategy  entails  an 
Increased  number  of  recruits  with  associated  Increases  in  recruiting 
and  training  costs.  The  second  strategy  involves  larger  expenditures 
on  bonuses  or  careerists'  base  pay  to  encourage  trained  individuals 
to  remain  with  the  Navy.  Both  strategies  cost  money.  The  question 
is  which — or  what  mix — of  the  two  strategies  eliminates  the  shortage 
of  petty  officers  at  least  cost.  Moreover,  since  the  two  strategies 
involve  different  mixes  of  careerists  and  recruits,  the  strategies 
may  have  different  implications  for  the  effectiveness  of  the  force. 
Indeed,  our  findings  suggested  that  first-termers  are  considerably 
less  productive  than  specialists  with  4  years  of  experience. 

Several  other  efforts  at  the  Center  for  Naval  Analyses  have 
addressed  these  questions  [11,  12].  Overall,  policies  directed 
toward  Increased  retention  have  been  found  to  be  the  most  efficient 
method  for  alleviating  Che  petty-off leer  shoftage:  costs  of  first- 
term  pay,  recruitment,  and  training  wete  considerably  larger  than  the 
expenditure  (via  reenlistment  bonuses  or  Increases  in  the  base  pay  of 
careerists)  necessary  to  improve  retention. 

6.  CONCLUDING  COMMENTS 

The  U.S.  military  provides  a  considerable  amount  of  training  to 
its  personnel.  Much  of  this  training  is  formal  and  takes  place  in 
the , classroom,  while  the  remainder  is  Informal  and  takes  place  on  the 
job.  We  have  adequate  information  on  the  costs  of  formal  training 
(although  there  1 &  a  clear  need  to  maintain  the  type  of  data  bases 
required  to  carry  out  the  ambitious  research  agenda  outlined 
earlier) .  Where  our  basic  information  is  inadequate  is  in  the  area 
of  estimating  costs  of  training  done  on  the  job.  This  paper  has 
discussed  one  such  effort  (the  EUS  survey).  We  used  the  supervisors' 
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responses  Co  compare  Che  coses  and  benefics  of  formal  schooling  and 
on-the-job  Crainlng. 

In  general  our  analysis  of  Che  supervisors'  answers  in  Che 
survey  provided  very  plausible  informacion  on  Che  effecciveness  of 
crainlng.  Our  major  caveac  is  ChaC  Che  survey  design  normed  Che 
effecciveness  measure  aC  che  level  of  Che  4-year  specialise;  Co  Che 
excenc  ChaC  learning  concinues  afCer  4  years,  and  parCicularly  if  le 
concinues  aC  differenc  races  for  Che  Cvo  crainlng  modes,  our  conclu¬ 
sions  mighc  be  differenc.  For  such  an  investigation,  however,  more 
daCa  need  Co  be  collecCed.  We  believe  ChaC  an  efforc  Co  collect 
supervisory  assessments  on  both  first-  alnd  second-term  personnel 
would  be  worthwhile. 

In  summary,  Co  answer  questions  about  how  Che  military  should 
Crain  ics  personnel  and  how  it  should  mix  trained  and  newly  recruited 
personnel,  much  more  research  on  training  effectiveness  is  needed. 

If  we  are  Co  address  effectiveness  questions  with  Che  degree  of 
sophistication  with  which  we  address  recruitment  and  reCencion 
questions,  we  need  Co  think  very  hard  about  what  data  we  need  and 
whether  experimental  data  are  required.  A  substantial  research 
agenda  remains . 
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REGRESSION  COEFFICIENTS: 

THE  GROWTH  OF  PRODUCTIVITY  OVER  4  YEARS 
AT  THE  FIRST  DUTY  STATION 


Table  A-l  provides  regression  estimates  for  the  growth  of 
productivity  over  time.  Separate  estimates  are  given  for  each  of  the 
12  Navy  occupations  examined. 


TABLE  A-l 


REGRESSION  COEFFICIENTS:  THE  GROWTH  OF 
PRODUCTIVITY  OVER  4  YEARS  AT  THE 
FIRST  DUTY  STATION 


Mess  Management  Specialist  Aviation  Machinist's  Mate 
_ (MS) _ _ (AD) _ 


A-school 

Direct  duty 

A-school 

Direct  duty 

Constant 

-1.09 

-19.30 

-17.72 

-45.46 

Months 

4.14 

4.42 

5.39 

5.89 

Months  squared 

-.045 

-.045 

-.062 

.066 

Machinist's  Mate 
(MM) 

Radioman 

(RM) 

A-school 

Direct  duty 

A-school 

Direct  duty 

Constant 

-20.70 

-34.97 

-21.83 

-45.27 

Months 

4.96 

4.96 

5.67 

5.94 

Months  squared 

-.053 

-.050 

-.067 

-.067 

. 

Electrician's  Mate 
(EM) 

Electronics  Technician 
(ET) 

A-school 

Direct  duty 

A-school 

Direct  duty 

Constant 

-11.62 

■  -38.77 

-23.44 

-47.64 

Months 

4.80 

4.82 

5.79 

5.48 

Months  squared 

-.052 

-.049, 

-.068 

-.060 

Constant 

Months 

Months  squared 


Aviation  Electrician's  Mate 
(AE)  . 


A-school  Direct  duty 

-30.26  -56.37 

5.85  5.82 

-.065  -.061 


A-2  •  ' 


TABLE  A— 1  (Coat’d) 


_ Nuclear  power  occupations _ 

Dental  Hospital 

Technician  Corpsman  Machinist's  Electrician's  Electronics 
(DT)  (HM) _ Mate _ Mate _ Technician 


Constant 

12.24 

-11.17 

-29.63 

-20.48 

Months 

4.51 

5;54 

5.33  - 

5.73 

Months  squared 

-.056 

-.067 

-.058 

-.068 

-29.33 

6.46 

-.079 


NOTE:  In  the  regression,  months  are  measured  as  months  at  the  duty 
station.  Since  enlistees  in  the  different  occupations  train  for 
different  amounts  of  time  before  arriving  at  the  duty  station,  it  would 
be  misleading  to  directly  compare  the  regression  coefficients  across  the 
occupations.  For  example.  Nuclear- Power  Electronics  Technicians  have 
been  in  the  Navy  18  months  before  they  arrive  at  a  duty  station  (versus 
2.5  months  for  direct-duty  Electricians'  Mates). 
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PIPELINE  COST  CALCULATIONS 


This  appendix  details  the  procedure  followed  to  compute  A-school 
costs.  All  data  are  for  FY  1979  (pay  table  In  effect  1  October 
1978).  The  training  data  are  derived  from  the  Navy's  Training 
Analysis  and  Evaluation  Group  (TAEG)  and  include  the  costs  of  formal 
training  and  wages.  All  personnel  are  assumed  to  progress  in  pay 
grade  at  the  sake  rate:  average  pay  grades  are  1.81,  2.76,  3.51,  and 
3.98  for  the  first  4  years,  respectively.  The  lengths  of  Individual 
courses  and  their  costs,  calculated  from  FY  1979  TAEG  data,  can  be 
found  in  [6], 

Some  occupations  have  parallel  training  pipelines.  If  there  was 
not  a  dominant  pipeline,  we  constructed  average  training  costs  by 
weighting  the  costs  of  the  parallel  courses  by  their  enrollments. 

For  example.  Machinists'  Mates  take  either  Course  Data  Processing 
(CDP)  Code  6492  or  6493  for  the  final  course  in  their  training 
sequence.  In  FY  1979  there  were  2,414  enrollments  in  CDP  6492,  and 
2,394  enrollments  in  CDF  6493.  Costs  (lengths)  were  $1,301  (39  days) 
and  $989  (29.  days),  respectively.  The  average  cost  and  length  of  the 
final  course  in  Che  Machinist's  Mdte  sequence  is  thus  computed  as 
follows: 

Average  cost  * 

Average  length 

Aviation  Machinists'  Mates  have  17  parallel  courses:  here  we 
used  the  dominant  pipeline  course.  For  Electronics  Technicians, 
however,  the  four  parallel  first  courses  display  relatively  even 
enrollments;  thus,  in  this  case  we  weighted  both  the  costs  and  course 
lengths  by  the  enrollments. 

Data  for  each  occupation  examined  are  given  in  Cable  B~l. 
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